1. Field of the Invention
The present invention relates to a display screen allowing for recordation of a fingerprint placed anywhere on the display screen, and a method to record a fingerprint anywhere on a display screen without affecting the display function. The present invention enables a cellphone to read a fingerprint when it is placed anywhere on the display screen. A cellphone maker no longer needs to allocate space on the front or back of the cellphone for a discrete fingerprint reading device.
2. Description of the Related Art
Identification is important issue in our digital, fast moving world. The problem lies in how to securely identify people. Credit cards can be lost or stolen. Picture identification cards suffer from inattention by the people that guard the door. Several years ago a top secret facility had several of the employees replace the pictures on their badge with cartoon characters (Micky Mouse, Daffy Duck etc.) who then successfully entered the top secret facility for several days. The fundamental issue is to assure that the identification card or credit card actually belongs to the person representing themselves to make a purchase or to gain entry.
Fingerprints have been used for identification through much of recorded history. They are found on Babylonian clay tablets, on the walls of Egyptian tombs, on Minoan, Greek, and Chinese pottery, as well as on tiles from ancient Rome. Many of these fingerprints were likely deposited unintentionally, some are decorations, but researchers believe that some fingerprints found on pottery were impressed so deeply and deliberately that they were meant to uniquely identify the artist or owner.
Several different styles and types of fingerprint readers exist, but all share the same goal, that is, to accurately record the unique features defined by the friction ridges (and valleys) on the finger. Fingerprints are identified by three levels of features. The flow of the ridges (Level-1) is generally classified as an arch, loop, or whorl. Level-2 features describe significant changes along individual friction ridges—primarily bifurcations and endings. These Level-2 features are referred to as minutiae and are the primary means of identification in current implementations. Features present within and between the friction ridges are referred to as Level-3 features. Level-3 features include pores, scars, width changes, shape changes, creases, breaks, etc.
A modern fingerprint sensor is an electronic device used to capture a digital representation of a fingerprint, that is, a fingerprint image. Many competing technologies exist for collecting fingerprint images—pressure sensors, capacitive sensors, optical sensors, and thermal sensors, to name a few. While the “raw” captured fingerprint image can be stored for general pattern matching, it is common to digitally process the raw fingerprint image and create a more efficient biometric template (a collection of extracted features) which is stored and used for matching. Whatever physical properties are used to capture the fingerprint image, it is critical to collect a high-quality (clarity & contrast) image of the fingerprint because the image quality is highly correlated to overall fingerprint system performance (see NIST 8034 Fingerprint Vendor Technology Evaluation [FpVTE2012]).
If the goal is to sense a fingerprint placed anywhere on a display screen, then a buried sensor array must exist across the entire display screen, and the costs and complexities are high. The most commonly used sensing technology is currently capacitive, and sensing electrical charges is most effective when the sensor array is located very close to the surface of the splay screen. This drives unrealistic material thicknesses and puts the sensor array in front of the LEDs and interferes with the display function.
Direct axial/optical solutions solve issues involving close proximity to the screen, but retain the need for a large, high-density sensor array, and also require micro-lensing and increased thickness. They must see “through” the regular illuminating layers of the display so that requires special materials with unique optical and electrical properties to be used in various layers throughout the display.
Optical fingerprint imaging involves capturing a digital image of the print using visible, UV, or Infrared light. This type of sensor is, in essence, a specialized digital camera. In most implementations, the sensor is built with a clear touch plate onto which the finger is pressed. Under the touch plate, a light source and a camera sensor are arranged strategically with various optical elements to focus a clear, high-contrast image upon the camera sensor.
The procedure used by all modern fingerprint scanners to capture a fingerprint using a sensor entails sliding, rolling, or touching the finger on a sensing area which, according to the physical principle in use (in this case, optical), captures the difference between valleys and ridges. It is important that the optical elements of the image capture de-vice preserve a clear, accurate, and high contrast representation of the fingerprint to achieve a high signal-to-noise ratio (S/N). One physical phenomenon often used in optical fingerprint readers to increase contrast and S/N ratio is Total Internal Reflection (TIR)—and an ancillary property—Frustrated Total Internal Reflection (FTIR).
Total Internal Reflection is an optical phenomenon that occurs when a ray of light strikes a boundary at an angle larger than a particular critical angle with respect to the normal to the surface. If the refractive index is lower on the other side of the boundary and the incident angle is greater than the critical angle, and all of the light is reflected back into the original medium. This can only occur where light travels from a medium with a higher [n1=higher refractive index] to one with a lower refractive index [n2=lower refractive index]. For example, it will occur when passing from glass to air, but not when passing from air to glass. The Critical Angle is the angle of incidence above which total internal reflection occurs. The angle of incidence is measured with respect to the normal at the refractive boundary.
An important side effect of total internal reflection is the propagation of an evanescent wave across the boundary surface. In TIR conditions, although the entire incident wave is reflected back into the originating medium, there is some penetration into the second medium at the boundary. This wave in the optically less dense medium is known as the evanescent wave.
If a third medium with a higher refractive index than the low-index second medium is placed close to the interface between the first medium and the second medium, the evanescent wave will pass energy across the second into the third medium. This process is called “Frustrated” Total Internal Reflection (FTIR). The FTIR phenomenon occurs only when the spacing between the two higher index media is small (on the order of 10s of nanometers). The dimensions of the ridges and valleys in a fingerprint are larger than the spacing necessary for the interaction that creates FTIR. Thus, as a finger approaches and touches a glass plate, light is absorbed and re-radiated in all directions where the friction ridges touch the glass, but where the valleys have a few tenths of a millimeter above the glass, all light rays that strike the surface of the touch plate above the critical angle are reflected.
When the touch area is viewed below the glass (or other transparent cover) of the display screen, the areas where the ridges touch the glass are one color and intensity, and the valleys are another. The ridges can appear dramatically lighter or darker than the valleys depending on the orientation of the light source and the viewing angle. In either case, this resultant high-contrast image is ideal for the typical digital-camera-based optical finger-print readers.
A simple, common characteristic of most fingerprint sensors, whatever the method of operation, is that the finger must contact with the sensor device directly. For the cellphone, application in particular this is a significant disadvantage because cellphone users prefer to use a fingerprint reader on the same surface that they view (the screen side). Users also prefer large viewing areas. However, all current “screen side” cellphone fingerprint reader solutions require space on the face of the cellphone exclusively for the fingerprint sensor.
In addition, there is a strong user and manufacturer preference to have a uniform cover glass over the entire face of the cellphone. In fact, some cellphone designers are attempting to encapsulate the entire surface of the cellphone in glass. The strength, scratch resistance and stiffness of the glass make it an advantageous surface material.
However, because all current fingerprint sensors (which are appropriately sized for use in cellphones) must be in direct contact with the finger, the cover glass must have a hole for the sensor to fit. The hole adds cost, presents a perimeter that must be sealed from the environment, and creates an area of weakness in the glass. Cellphone manufacturers are seeking a fingerprint sensor technology that can be placed under the cover-glass to avoid creating the hole.
An alternative approach is to scan the illumination and direct the resulting reflected light to a sensor. The power detected by the sensor can then be used to recreate the image by scanning a display screen and changing the intensity of the display to reflect the power detected by the sensor. This technology is commonly used in the scanning electron microscope and many confocal microscope designs because the image is free of distortions caused by focusing the image, the point of view of the image is from the illumination source and the depth of field is enormous compared to other imaging techniques. Relative to fingerprints, this method of creating a fingerprint image was used in U.S. Pat. No. 4,553,837, entitled “ROLL FINGERPRINT PROCESSING APPARATUS,” by Daniel H. Marcus in 1983.
Another optical system for scanning a fingerprint is disclosed in Taiwan Patent Application 104208311. The '311 application discloses a system wherein the finger s illuminated from below the sensor and the image is projected onto a camera chip adjacent to the sensor. FTIR is used to enhance the image of the ridges so that the ridges are bright and the valleys are dark. If the sensor is optically connected to the cover-glass of a cellphone (by bonding to the cover-glass using optically clear adhesive with a matching index of refraction for instance) the glass becomes part of the sensor. Electronically, this invention focuses the image onto a digital camera, and the sensor pixels on the camera chip are scanned to create the digital image. The resultant image can be processed to remove distortions and identify the minutia in order to determine a positive identification of the fingerprint. The point of view of the image will appear as if viewed from the left. The image is naturally compressed along the length of the arrow but is full size in the third dimension. Care must be taken with the optics to ensure that the camera chip is maximally utilized. Even then, the image resolution in pixels per inch may not be symmetrical. The image distortions are geometry related and can be corrected to some extent software, but the resultant image is unlikely to be fully corrected and will not have a true one-to-one relationship to the original fingerprint. Also, manufacturing tolerances limit how thin the structure can be made.
Attempts to solve problems associated with sensing fingerprints through a display screen of, for example, a cellphone involve substantial changes to the core display materials. Many of these materials are expensive and capital intensive. The additional layers increase the thickness of the devices. The straightforward approach requires that density arrays of sensors be placed across the screen.
U.S. Patent Application Publication No. 2015/0036065 discloses using a layer of sensors under the display to allow fingerprints to be read all across the display without screwing up the way the phone looks when you're using it. This published application describes adding a layer “under” the display to read the fingerprint. This is different from the common approach used by most team working to find a way to take a fingerprint from the screen. Most teams are trying to find a way to add a layer “above” the display LED/LCD layers so the sensors can be very close to the finger. The trick is to make that layer completely transparent so the image from the display is not corrupted. That's been difficult to achieve.
With regard to early art on using the display to illuminate the fingerprint for reading, Apple has attempted to use the pixels in the display as “sensing pixels”, See U.S. Patent Application Publication No. 2015/0178542, entitled “Finger biometric sensor including drive signal level updating and related methods.”
U.S. Patent Application Publication No. 2015/0036065, entitled “Fingerprint Sensor in an Electronic Device” discloses sensing fingerprints directly on the screen from multiple fingers and mentions ultra-sonic sensing. While the ultra-sonic concept is only thing out there that is even remotely close to the present invention it is not very close except it likely doesn't require physical sensors to correspond 1-to-1 with the detail desired in the fingerprint image.
Still further, the size of the fingerprint sensor used on cellphones is determined by a trade-off of increasing reliability (which requires large sensing area) and decreasing cost (which increases with sensing area). Present sensors on cellphones are as small as possible while still providing sufficient reliability as required by the cellphone users, which is relatively low. However, in order to provide sufficient security for significant financial transactions the cellphone fingerprint sensor reliability will have to be at least as reliable as the identification chip system being introduced in credit cards. This will require significantly larger fingerprint sensor size—area that is simply not available on cellphones. By having the entire screen available to be a fingerprint sensor there is no longer a practical restriction on the size of the fingerprint sensor.
With the foregoing in mind, there are no existing full-screen fingerprint solutions at this time. All known potential solutions involve adding materials or layers to the display stack and they include thousands or millions of micro-sensors to measure the details of the fingerprint. The present invention avoids a issues of the prior art by dramatically reducing the number of sensors required and optionally allowing the sensor elements to be relocated from the area of the display to the perimeter and utilizing sequential energy pulses and accurate timing to build a fingerprint image. The device has the ability to scan the entire screen to detect the location of a finger in contact of the screen and measure the intricate features of a fingerprint. Through the remainder of this application, this system is referred to as the remote sensing fingerprint reader.